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Consider an observed process that is the sum of a Wiener noise 
process and the integral of a not necessarily gaussian signal process. 
The innovations process is defined as the difference between the 
observed process and the integral of the causal minimum-mean- square- 
error estimate of the signal process. Then if the integral of the ex- 
pected value of the absolute magnitude of the signal process is finite, 
we show that the innovations process is also a Wiener process. The 
present conditions are a substantial weakening of those previously 
used in which the integral of the signal variance had to be finite. The 
new result is obtained by using some recent results in martingale 
theory. These results also enable us to obtain similar results when the 
Wiener process is replaced by a square-integrable martingale. 

I. INTRODUCTION AND DISCUSSION OF RESULTS 

We shall be concerned with stochastic processes of the form 
dy = zdt + dw, i.e., y(t) = [ z(s) ds + w(t), ^ t ^ T (1) 

Jo 

where w ( • ) is a Wiener process with 

E[w(t)) = 0, E[w(t)w(s)] = min (t, s) (2)% 

and z(-) is a not necessarily gaussian process such that, for every 
0^ s < t ^ T 

w(t) - w(s) ii ff. = <j\y{r), ^ t ^ s\ (3) 
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* Later we shall replace w(-) bv a square-integrable martingale (cf., Theorem 
3). 
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where <r{ • } denotes the smallest sigma-field generated by { • } and, 
following K. ltd, the symbol u will be used to denote statistical in- 
dependence. In our earlier work, 1 we also assumed that 



E f | z(t) \ 2 dt< oo . 

Jo 



(4) 



This assumption not only enabled us to define the conditional ex- 
pectation, 

E[z(t) | ff t ] = z(t), say (5) 

but also to conclude that |2(-)| and |f(-)l" ( as wel1 as l 2 (")l and l 2 (")| 2 ) 
have finite integrals a.s. Then in Appendix I of Ref . 1 and independently 
in Refs. 2 through 4, the following result was proved: 
Theorem 1: Under the assumptions (1) through (4), the 'innovations' 
process {v(t), $,} where 

"(0 = V(t) ~ f *(*) ds, g t ^ T (6) 

J 

is a Wiener process, relative to the sigma-fields SF« generated by y(-). 
Moreover, v(-) has the same statistics as the original Wiener process w(-). 
The reasons for the name "innovations" and some applications of 
the concept are discussed in Refs. 1 through 5, and the references 
therein, so that we shall not pursue them here. Our interest will be 
in relaxing the conditions under which the theorem is true. In par- 
ticular we shall show that the condition (4) can be replaced by the 
weaker condition 

E [ | zit) \dt < oo . (7) 

Jo 

This is a substantial relaxation of (4), since now we can even consider 
cases in which z(-) need not have finite variance and need not be 
square-integrable in t a.s., (examples of this type are easy to find for 
gaussian z(-), cf., Ref. 6). 

Nevertheless, even this weakening is not the best possible. Such a 
claim seems surprising, since to begin with, the standard definitions 
(see e.g., J. L. Doob 7 ) of conditional expectation require that 

E\z(t)\ < oo. 

In our problem, however, L. Shepp has pointed out by example that 
even this condition on z(-) is not always necessary. 8 In Shepp's 
example, 



INNOVATIONS THEOREM 1489 

z(t) = 77, a Cauchy distributed random variable, 
independent of w(-). 
Then it is easy to verify that z(-) as defined below by equation (8) 
can be used to define the innovations process v(-), even though 
E I z(t) I does not exist. Nevertheless, Bayes' theorem shows that 

/ 77 exp r)w(t) - \ \p(n) dri 

m= r — r W~ "* (8) 

It is true that there is a generalized definition of conditional expec- 
tation (see, e.g., p. 342 of Ref. 9) that can be used when E | z | does 
not exist, but the exact sense in which equation (8) fits this generalized 
definition and the more general problem of trying to define i(-) 
under just the assumptions (1) through (3) requires further investi- 
gation. So much for the assumption (4). To see that equation (3) 
is also not always necessary, we can let z(t) = w(T), =i t ^ T. 
By using the results of Ref. 6, it can be shown that even in this case 
we can define a variable z(-) such that the process v(-) of equation 
(6) is still a Wiener process. Having discussed such examples however, 
let us quote precisely the results that we can prove. The first is: 

Theorem 2: Under the assumptions (l)-(3) and (7), the innovations 
process \p(t), JF,} defined by equation (6), is a Wiener process with the 
same statistics as w(-). 

Besides extending the range of applicability of the innovations pro- 
cess, Theorem 2 is of interest, at least to us, because of the relatively 
new techniques used to prove it. Thus Theorem 1 was proved in Ref. 1 
by using a martingale theorem of Levy and Doob (cf., p. 384 of Ref. 
7) to the effect that a continuous-path, finite-variance martingale with 
conditional variance equal to t must be a Wiener process. The assump- 
tion (4) that was made in Theorem 1 enabled us, with some computa- 
tion, 1 to establish the finite-variance and conditional variance proper- 
ties. In Theorem 2, the process z(-) may itself have infinite variance 
and we cannot easily apply the theorem of Levy and Dobb. Fortu- 
nately, however, some recent developments in martingale theory, due 
to H. Kunita and S. Watanabe 10 and to P. Meyer, 11 ' 12 have provided a 
generalized form of the Levy-Doob theorem that we can use to prove 
Theorem 2, and in fact with less computation than was needed in 
Ref. 1 for Theorem 1. Moreover, the results in Refs. 10 through 12 
have suggested our second result, which in fact includes the first one. 
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Theorem 3: Let dx = z dt + dM, where 

E C | 2(0 \dt < oo , M(t) - M(s) US. (9) 

Jo 

and M(-) is a square-integrable martingale. Then {/*(«, «), $t, P\, where 
M (0 = x(t) - f z(s) ds, i(t) = E[z(t) | <F«], 

•'0 

is a locally square-integrable martingale with quadratic variation the same 

as that of M. 

P. Frost has obtained the result of Theorem 3 under the stronger 
hypotheses that E\z{-) | 2 is integrable and M(-) is continuous. 13 

A special form of Theorem 3 was obtained earlier 8 by use of a result 
of Doob (See p. 449 of Ref. 7) that a continuous square-integrable 
martingale {M(t), 03,} satisfying 

E[[M(t) - M(s)f | (B.} - £J/ o | G(t, a,) | 2 dr | (B.j 

can be written as a stochastic integral with respect to a Wiener 
process. In other words, this leads us to extend equation (1) by 
allowing a stochastic coefficient for dw ; we may note that Kunita and 
Watanabe have shown that a (locally) square-integrable martingale 
of a Wiener process can also be written as a stochastic integral. 

Finally we may remark that the proof of Theorem 3 also yields the 
following. 

Corollary 4: Theorem 3 remains valid if hypothesis (7) is replaced by 
i(-) exists and ( z(t) dt < « a.s. (10) 

Jo 

Note that condition (7) guarantees (10), not just for conditional ex- 
pectations with respect to the sigma-fields {£,} generated by dx = 
z dt + dw, but with respect to any family of sigma-fields {A t \ ; clearly 
the condition (7) is too strong and does not, as we also noted earlier, 
exploit the special nature of our problem. We have as yet found no 
condition on z(-) simpler than (7) that will imply (10). It seems that 
what will be needed is a condition on x(-) rather than on z(-) and M(-) 
separately. In fact, we may observe that in all our examples [in the 
lines below (7)], the process x(-) is absolutely continuous with respect 
to a Wiener process; however, we have not yet worked out all the 
implications of absolute continuity. 
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II. PROOFS OF THE THEOREMS 

We shall begin with some definitions; general references for the 
following material are the paper of Kunita and Watanabe, 10 and the 
book 11 and lecture notes 12 of Meyer. 

All our random variables will be defined on a probability space 
(12, (B, P). We shall also assume that we have an increasing and right- 
continuous family, (B, , ^ t ^ T, of subsigma-fields of (B, each (B, 
containing all the null sets; we shall write (B r = (B. 

A process \M(t, co) } will be called an L 2 -martingale or a square- 
integrable martingale if M(-) has a.s. right-continuous sample paths* 
and if 

EM 2 (t, oj) < oo and E[M(t, a>) | (B.] = M(s, a>) a.s. 

Every L 2 -martingale {M(t, <o)} has associated with it an increasing 
function, called its quadratic variation [M(t), M{t)\, that can be 
calculated as (cf., p. 92 of Ref. 12) 



(ID* 



[M(t), M(ff] = lim £ [M(t i+1 ) - M(t,)] 2 

max If , + ,-f .■ |-»0 

where (t , h , " ', t„) is any partition of (0, t) and the convergence 
of the sum is in L 1 , i.e., in the norm \\A\\ 2 = E\ A |. When M{-) has 
continuous sample-paths, the quadratic variation is often written 
(M, M), and can be identified as the unique increasing process such 
that process M 2 {t) — (M(t), M(t)) is a martingale. 

A positive random variable t(o>) is called a stopping time if for every 
t > 0, the event {t(o>) ^ t\ belongs to (B, , i.e., it depends only upon 
the history up to t. 

A example of a stopping time is the first time (M{t)) exceeds a given 
level. An important property of a stopping time is that if Miff is a 
martingale process, then the "stopped" process M(tAr) is also a mar- 
tingale. 1 

A process M(t, a>), ^ t S T, is said to be a (continuous) local La- 
martingale, or a locally square-integrable martingale, if there exists 
a sequence of stopping times t x ^ t 2 ^ • • • , increasing to T such that 
the stopped processes are all (continuous) L 2 -martingales. The unique 



* Meyer (p. 45 of Ref. 11) has shown that if E[M(t, w)] is right-continuous (and if, 
as assumed above, {(B/| is right-continuous) then there always exists a right-con- 
tinuous modification of M(-). If the paths of M(-) are a.s. continuous, we shall 
call the martingale continuous. 

t This notation is a shorthand (cf. Meyer 12 ) for the usual procedure involving 
a sequence of partitions. 

* We use the notation a /\ b = min(n, b). 
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(continuous) increasing function lim,,-* [M(<At„)] will be called the 
function associated with the local L 2 -martingale M, or often, the 
quadratic variation of M. 

2.1 Example 

The ltd integral 

7(0 = ( /(s, co) dw (s, co) (12a) 

Jo 

can be defined as a continuous function oH,0 ^ t ^ T, for every /(• , • ) 
such that /(• , •) is independent of future increments of w(- , ■) and 

f f(s, co) ds < oo a.s. (12b) 

Jo 

If in addition /(•) obeys 

Elf I 1(s, co) | 2 ds] < oo (13) 

then the Ito integral I(t) is an L 2 -martingale and its quadratic varia- 
tion is 

[J(0, J(0] = W),I(t))= f 1\s,co)ds. 

Jo 

By defining 

min \t: J f(s,u>) ds ^ 

.T, if { • } is empty, 

we see that the Ito integral under equation (12) is always a continuous 
local L 2 -martingale, with quadratic variation 

[M(t), M(t)] = lim ( " f(s, co) ds = [ f(s, co) ds. 

n-oo •'O J 

2.2 Proof of Theorem 3 

We note first that, with z(t) = z(t) - &{t), 

m(0 = 2/(0 - [ z(s) ds= [ z(s) ds + M(t) 

Jo J o 

is a martingale relative to the fields {ff,} because for t < t, 



T„ = 
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E[v(t) | JF T ] = v{r) +E\J [2(«) rfs + dM(«)] | 5F T ], 

= v {t) + £ £[z(s) | ff. | 8FJ rfs + E[M(0 - M(t) I JFJ, 

= v(t) + f Ods + = v(t). (14) 

The interchange of £"[• ] and the integral is justified by Fubini's theorem 
and the assumption (7). 

Now define a sequence of 5, -measurable stopping times by 

T n = [inft : t ^ 0, M (0 ^ »}A7\ (15) 

Since /*(£) < oo a.s., the {T n } increase to T. Also it is clear that, for 
each n, fi(tAT„) is a square-integrable martingale. Therefore, /x(-) is 
a locally square-integrable martingale. To find its quadratic variation, 
we first show that 

WtATJ, m(«AT.)) = [M(tAT n ), M(tAT n )]. (16) 

To obtain equation (16), we use the definition (12), the fact that 
[ • , • ] is zero for a continuous process of bounded variation, and the 
inequality 

£ [ M (/, +I ) - nW>T < E [/ ,, * ,f w] i + ^ [M(t M ) - M(t<)] 2 . 

Finally by letting n tend to infinity, we see that 

[ M (0, m(0] = WO, M(0L ogfgr. 

This completes the proof of Theorem 3. 

2.3 Proof of Theorem 2 

When M(t) is a Wiener process, its quadratic variation is t. To 
obtain Theorem 2 from Theorem 3, we now use a theorem of Levy and 
Doob (See p. 384 of Ref. 7), as extended by Kunita and Watanabe 
(See p. 217 of Ref. 10), that a continuous locally square-integrable 
martingale with quadratic variation t must be a Wiener process. 

It is interesting to point out that the Kunita-Watanabe proof of the 
extended result is also considerably simpler than the original proof be- 
cause the new proof utilizes the powerful new tools of the stochastic 
integral for Lo-martingales and the Ito differential rule for such 
processes. 
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2.4 Proof of Corollary 4 

We note that hypothesis (10) suffices to ensure that /*( • ) is a.s. finite, 
so that the {T n } of equation (15) are well defined and tend to T, and 
also to ensure that p,(tAT n ) is a square-integrable martingale. These 
are the essential ingredients of the proof of Theorem 3. 

The present wider conditions under which we have been able to 
establish the innovations result will, of course, extend the range of 
problems in which the innovations can be used. In particular, we have 
applied Theorem 2 to show 14 that a general likelihood-ratio formula, 
derived in Ref. 5, for processes obeying equations (1) through (4) 
remains valid if equation (4) is replaced by the weaker conditions 
that 5z 2 {t) dt < oo a.s. and J"S \z(t) \dt< oo; 
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